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, _ _ . _ .2^ 

; . Analysis o£ covariance is often cohceptuaiized as an analysis 
o£ variance o£ the residual ^scores that are obtained when the 
dependent variable is regressed on the covariate. Although this 
cdhceptualization is "intuitively appealing, it is mathematically 
incorrect. If residuals are obtained from the^ pooled withih- 
groups regression coefficient (b;^) , an analysis of variance on the 
residuals results in an inflated a-level. If the regre^ssion 
^^eff i cienj:_ f .ti^e^ i6-t^J_^ample-c<5mb-i-ned int o— one-^group (b'j) rs 
used, .ANdVA on the residuals yields an inappropriately conserva- 
tive test./- In either case, analysis of variance of residuals fail: 
to provide a correct test, because the significance test in analy- 
sis of covariance requires consideration of both b^^ and bj,*uiililce 
analysis; o^residuals . It is recommended that the significance 
test of treatment^ effects ' in analysis of covariance be conceptual- 
ized as a comparison of models whose parameters are estimated by 
the principle of least scjuares • 



Ahaiysi^ ,o£ covariahce (ANCOVA) tends to be one of the iiiost iiiis- 
understood and misused statistical techniques, partly because is 

-requires a, synthesis o£ multiple regression/ correlation ^ ^^^^^ -and =:^- 
ahalysis of variance (ANOVA^ concepts. One .commonly employed method 
6£ explaining how ANGOVA relates to MRG arid ANGVA is to make use o£ 
the concept. of a residual ^cofe, ANeOVA is presented as ah ANOVA on 
the residual scores that are obtained when the dependent variable is 
regressed oh the covariat.e. For example^ Marascuild states that "Gb- 

Xar i an.c.e_ad3aas tmen-t - i-s -equ i varentr-lrb' p r 03" e cTihg the' eTrneci score [ the 
dependent variable in his example] in a directidh parallel to. the 
regression line to the IQ score [the covariate in his example] defined 
at X [where X is the grand mean f^r the cbvariate] . This parallel- 
projection is performed for all pairs of observations and an analysis 

of variance is then performed on the adjusted scores" (1971, p. 499). 

" _ - _ #^ 

This sort of explanation of ANGOVA can be found in many^ if not 

most, of the standard sourcebooJcs on statistics used by social scien- 
tists. To cite 'only a few, Snedecbr and Go ch fan state that "The 
analysis of covafiahce is essentially an analysis, of variance of the 
quantity-CY-bX)" (1967, p. 424}.. Gohen and. Gbhen say that "The AGV 
[ANtOVA] involves the analysis b£ (the residuals of J Y when one of ^ 
more other variables (the cov^fiates) have been partialled out. . . 
In ACV, the residual that is analyzed is.Y - Y^ for each subject . . . 
in exactly the same way as Y itself is analyzed in AV [ANOVA]" 
(1975 ," p^-"3d^)T Ac cb'f "di rig ~tb the SPSS "manual , '^Regression pf bced- ' 
ufes afe ^used to remove variation in the dependent variable due to 
one or more covariates, and a conventional analysis of variance, is 
then performed on the 'corrected' scores" (Nie, Hull, Jenkins, Stein- 
hrexiTipry »id fiehti 1975; p; 469). Similar statements can also be 



ioxmc! in such sources as Dixon and Massey (1969, p. 223), Kerlihger 
ana P-dhazur (1973, pp. 265-267), and tindquist (1955, p. M8) . 
Kuwever, it is well -known by these authors arid others that ANCOV^ 
an^ an ;thalysis of variance conducted on the residuals C^^^reafter 
called A>^ORES) will be slightly different^ becaus^ one degree of 
freedbm is U5ed to estimate the slope ^of the regression line of the 
dependent variable on the covariate. Thus^ without this adiustment 

in degrees of freedoiu., t he error t erm for A^ORES_jtfoj^ 

additional but artificial degree of freedom. With this adjustment, 
.it seems that ANGGVA and AN0RES should be identical.. In fact, how- 
ever, they are not identical even after the degree of freedom; adjust 
ment; The commonly held belief that they are? the same is a'reflec- 
tion of the lack ox understanding of the ANeSVA model; 

The current paper will employ the approach of comparing models 
using least squares (see, for example, Cramer, 1972) to explain why^ 
ANCdVA and ANORES are indeed different. Viewing ANGDVA as a com- 
parisbh of models considerably illuminates the logic behind ANCOVA. 
A numerical^ example illustrating^ the general principles will also 
be presented. — 

•'The analysis of ebvariance. test of " group differences can be 
conceptualized as the comparison of the following two models: 

II. Y^. = y + BXjj * z^. (2] . 

where Y^j is the score on the dependent variable d£ the ith subject 
in the. 3 th- group^ y is a grand nieari parameter, dj is a pirameter 
indicating the effect of the treatment, 8 is a regress ion'coeffi- 
cient, Xjj is the score oh' tho covariate for the itji subject in the 

B : 
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j^h group, and "e^^ Is an "error term for the itE subject in the 
jiE group, the inddels are compared By using least squares 'to esti- 
mate each parameter in eadh model and .then . comparing the error stun 
b£ squares (SSE) . for the' two models.- With the usual ANCOVA assump- 
tions (see, for. example. Glass , Peclcham, and Sanders , ° 1972 , br 
Elashoff, 196S) , the following expression has a central F distribu- 
tibh with k-1 and N-k-1 degrees of freedom if the null hypothesis' 
_oiia^€at-inen-^e^f^ec1:s ^s rrue theliumber of treatment groups, 

and N i£ the total sample size): - 

0 . 

F = . ^ ^ ^^^^ - aii/j£- n: (s) 

SSE (i)/eN-k-i3 , : • ;. 

The model comparisons approach makes clear that the F test iil • 
ANCOVA is a function of the extent to which scores on the dependent 
vari^le can be more accurately : predicted if group, membership is 
known than if it is not, where prediction is performed in both 
models by using least squares. Of crucial iSiportance later in our 
argument is the estimation of B in models I and II. A common mis - 
conception is that since both models contain a 3 parameter, the 
least squares estinrate "of g in model I will be identical to the 
least squares estimate of B in model H . However in model I the 
estimator is b^^^ the "pbcied within- groups regression coefficient f br ° 
Y regressed oh^K, while in model II the estimatbr is bj, the 
fegfessioh coefficient for Y. on X fbr the entire sample of obser- 
vatioiis 'combined into one grbup. In the absence--b£ group mefflber- 
ship parameters, bptimal prediction is obtained -by using by as the 
slope coefficient; whereas when different intercepts fbr each group- 
are allowed for by the introduction bf grbup membership parameters 



but a common ; slope of Y on X is .assuiiiedi dptimai prediction is: 
obtained by using b^^. It should be rioted tliat rarely will b^^- = by, 
principally because o£ sampling variability.- Note' also that even 
the corresponding g parameters, being estimated (i.e.^^ in Equations 
1 arid 2, respectivelTi— srre themselves unequal unless the :^null 
hypothesis is true or assignment to gioups is raridom; This cari be 
seen by thin&ing b£ _the -ANCOV regression models and 

: _•___•■_ _' o 

realizing^ that the value of a parameter remains :u:\chariged when 
\pother parameter is added to the model if arid only if the second 

parameter equals zero or.^the variables associated with the parameters 

. . ___________ ____ ____ _ " 

are uncorrelated. If the null hypothesis is true, the otj parameters 

in model I will be zero and hence the 3 parameters in models I and 

. rr^will be equal. If ^assignment to groups is random, the X variable 

will be uricoi related 'in the population with the ctj group -membership. 

variables and hence the. population regression coefficient associated 

with X will be the same in both models. 0£ course ^ even if orie or 

both of theie conditions are met with the result that the. g *s are 

equal, the values of b^^ arid hj will almost certainly differ because 

of sai^pling error. |' 

Certain authors~l¥".'g.'. Wirier, 1971^ pp. '753-76:4; see also Evaris 
and Aaastasib, 1968) have mistakenly stated that the equivalence of 
the two parameters is an assumption' of . the analysis of covariarice. 
However ^ ANCOVA does not in general require equivalence, although 
iiterpretatiori of results is clearest when groups are randomly con- 
stituted and hence the two g parameter values are equal. 

This distinction between h-^ and b^, as well as ■ the introduction 
of btj, the betweeri-group-S— r-e-gress-i-eri of 7 . ori X . , bfteri is difficult 



to explain when relying on extensions b£ the ANOVA approath; and 
hence contributes to the confusion surrounding ANCOVA. In contrast^ 
the. model ebmparisons" approach making the .utilization of least 
squares explicit shows why it is necessary -to define both b^^ and b^ 
in tjrder to discuss ANSOVA. However, bg is hot a least-squares 
estiinatdr of a parameter in any mod'?!, and seems to be of limited 
value, except when multilevels of unit of analysis are considered.-* 
For example^ analyses mig^ht be Icohducte^ both at tlie level of stu- ' 
dents .and classrooms, in which case bg may be of interest. For 
further discussion, see Burstein, Linn^ and Capell (19783. 

AN0RES can also.be conceptualized in terms of model cbmpari- 
sons. One virtue of tills approach is that it necessitates explicit 
consideration of how the residuals are to be obtained, be!^use 
either, bj^ or b^ could be used to define a residual score. Gbhen and 
eohen (1975, p. 308;) • have argu^sd : that b^^. shbuld'be used. However', 
'because none of the other previously referenced sources have stated 
which coefficient should be used, we will examine both ^ beginning 
with b^^. Tfi^ residukl score for the ith subject in the jth group 
c^an then:.^e.^wrdtten as Yjj^ -—b|^X^j ; And, the models compared by 

ANORES may then be written: 

> ■ ■ . ■ 

^i3 Mij V^'j ^ ■ ' ^ 

.^ij - Vij " ^ Hy " ~ "'(S) >^ ' 

The significance test is again obtained ioy comparing the error sum 
of squares for the two models as f ojLlows : 

P^^__ ^SSE (IV) - SSE-4H-IH^(^<Lr — - • 

• ■-. . ssE (in)/(N-k-i) . ~- 

The term N-k-1 appears as the ^denominator degrees of freedom because 
8 has been estimated to obtain the residual scores. 



the relevant question- at this point is how this F test relates 
tcHehat in Equation 3 from.ANCOVAj which translates to, how the errors 
associated with the models compart. Firsts consider the relationship 
%tween models I and III; it can Be shown that SSE^I), equals' SS^ 
^.'fdr ANCOVA, "i .e. , the adjusted, within-group sum of- squares, whicfc-— - 
in turn equals (see. Kirk, 19^^ p. 461^-:—— ~ ~ 

The error siM b£ squares for model II-I is given by SS^ .on the 
dependent variable' Y,/ - b^^X- ^ . Since . / 



arid 



ssEtiii) £2cy„ - 6„x„ ^ (7.3 - b„t m * 



algebraic manipulation -leads^ to 

SSECIIir ^. ZZCYj. - 7 j)2 _ ^2 2E(3Cij - ^.3)2 ^loj 

- = _ SSE(l) ; / . : • - - — - 

Consider next the relationship betw^^ri model II and model IV. .' In 
model II,*estimates for y arid 3 are arrived at ?o as to minimize 
the error sum d£. squares for such a two parapet er*modei% Iri model 
IV, however., b^ is fixed, "so that only y is es'timated through least 
squares; This estimate is, given by _ 



- y- '= - b^X..! . (11) • - 

it-must he the case_that^:SSE (IV) SSECI T) , because leas t squared 
for model II could always "chpbse'V^p to be as iri (11) and g = b^, 

icating the estimates of model IV- otherwise, the estimates b£j. _ 
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model H will diffe'f and provide a. better fit to the data^ yifeldihg. 
z smaller error sura b£ squares ^ since this is precisely the goal of 
the least squares procedure.._,Re£ererice to the formulas for the F ; 
test in,J«»f€<3VA"(^^ 3) aiid ANORES (Equation 6) reveals that the 

observed F ratio for ANQRES must be at least as large as, the F for" 
ANCOVA, since SSECI) = SSECIII) but SSECIV). > SSE;(I'lf. The extent 
of the discrepancy will depend on the extent to which b^J' differs 
from b^,\ since it can be shown that 

SSE(IV) = SSECII^ + (bw-by}^ 2E(Xij-^.;32 _ . ^j^) 

This relationship shows that the test given in Equation 6 is 
not a l^itiiate F test, because the sampling distribution of the 
statistic differs systematically ff^om the sampling distribution of , 
the proper test^tatistici The reason that Equation 6 is iirapprb- ' 
pfiate is that the nximerator expression is £not-^^^ 

^^'^^^'^''^ of freedom; . This is 

obvious since SSE (IV) - SSECIII) is systematically larger than 
SSECII) - SSE(I) , which (when .divided by is "dis^ribuied-as^a""^ 

chi-squarf ^ith k-l degrees of /free(gm_._3ence, ANOl^SlIs^ 
.obtairiL-residuai- scores will lead to' an inf lated- a- level , and is 
ifcertainly not equivaJjsnt to ANCOVA, 

Although^the use p£ to obtain residuals does not reproduce . ' 
ANCbVA^' it is also possible to perform ANGRES using b^ to obta^in 
residuals/. Perhaps it is this form of ^ ANORES that previous apthors 
have had in mind when they have written of the equivalence between : 
ANCOVA and ANORES/ It should be noted that this ^ppio^cS-^-irs^oTtenT' 
referred to as. a residual gain analyser '(e.g;", Gorder-Bolz, , 
' . • . ' ' . " . - • ■ - • • \- 

« V 
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With this method of forming res iJduai scores; the dependent variable 

; for the ith- subjee;t in the 3th- group is Y^4 ^1X13 • Models to be 

compared ace then: ^ ' 

V. - bjXi^ = u ^ a^. + e^j- ' j^l3-j 

(14)' 



VI. 



Y-z - b^X^H = u + e^,' 



Once again, we might pei-Jorm i significance test by_^comparing the 
error sum of squares for the .two. models as - - • 



F = 



(15) 



SSE(V}/(N-.k-13 i 

As before, N-1c-l appea^s_i^i--i:Ke^denominatbr because h^ been 
climated -to^btain the r^siduajE--s^oxgrr''"'1j^^ I, II ^ V^ and 

VIjnust__be_coji^aT-gaT^M6d^^^^ II and VI are identical because both 

include hj as- the slope value and y = f. bjl... in both cases; 
Hence , ; : . 



■■ ssECvi) = ssE(ii) 



(16} 



-^onsider next thi? relatiOriship^between models I and- vT^^n^model iV 
estimates for otj , and B are chosen so as to minimize the sum Of 
squared errors, by the definition of leas-t squares. In model V, 
least squares estimates -are obtained for p and aj subject to the 
const-raint- that 3 = b^. However, b^ is the least squares estimate 
and therefore must lead t6_a minimal S3m-4>-f'-^qUared e Hence, 
smSl^^il^T '(17) 



li the extent Of the difference, in error sum of squares is delated 
to the difference between b^ and bj. In particular, 

^ SSE(V) = SSE(I) + (% bx)2 £2j;xij-I^)2. ^^g^ -. 
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Referring to Equant iDn$^3. and 14 shdws that the Vf^ yalue'' (>fetaihec 
by using this apprafbh to ANORES must result in a' value that is 
less than Qi^qual to the F obtained with AiMCOVA, with equality 
holdiB^'only i£ b^^ = bj. Thus, this form d£ ANORES^alsb fails 1?c 
/H5e^equivalent to ANCOVA arid fails tg provide a valid F test, for 
iiiuch the same reason as did ANORES with b^^. With the bj approach 
however, neither the numerator noV the. deupmiriatbr have th^e ehi 
square. distributions necessary to make their ratio an F random 

■ - > " 

variable . • , , ; 

Thus, claims that ANORES and ANCOVA are equivalent are false 
whichever approach to ANORE.S i^s e^^loyed/ The fact that models I 
and III are equivalent arid that : II and Vi are also equivalent ' 
suggests -that it is possible "to duplicate the ANCOVA test by exam 
iiiing residual scores. Specificafly, the following - test is equiv 
lent to the MCOVA test: 

(SSE (VI I - isE ( 1 1 L) Hj(^ 



SSEeiII}/CN-k-13 

the crucial fact is that ANCOVA depends upon both b^^ and b^, and 
corise'queritly so must ah equivalent analysis of. residuals. It is 
insiifficient to atXend to -only one regression coefficient. As 
stated previously, "we belieye that this is one of .the least under- 
stood points cbncerhihg ANCGVA. Only by close inspection of jndde. 
and the, least squares" principle does the logic underlying the reg: 

sidn coefficient parameter become clear. 

.•^ ' ^ " ■ ' ' ■ • 

. _ _ . ^ - ^ • 

A ntnnerical exanrple will demonstrate the theoretical argumeni 

*j_ _ • . ' •_ _ 

concerning the relationship betweeh'^ANCOVA arid ANORES. , Corisider i 

hypothetical data g'iveri-iri table 1. The error sum_o-f---sxrffares' for 



V^; $h*e six models previously outlined are presented in Table 2 Tor 
tfiese data, and Table 3 presents analysis -^D-f- varia^ tables for 



■ ^^NCOVA^ndr -the two forms of AWQRES^^ iResuits' here verify thg^t the 
' ■ - -^ - - - _ .J . _• - ^ 

insert Tables 1, 2, 3,.and:4 about here ' 

"F" obtained with ANORES usihg. b^v to form residuals is tod large;, 
.. while the "F" when bj is used is too small. Thi^' example illus- 
trates that even when the two groups being cenpared have similar' 

CJ- - * ■ 

..distributions on the covariate, the use of bw al6he\can lead to an 

inebyrect conclusion _of _sit.ati5^t4rcar-s:ignxflcance at ' 

— *^ IVith different data, the use of "bj aldne might result in a failure 
^ to recognize an appropriately si-gnificaiit restt3:t. For example, by 

simply revising the data from the" first example by subtracting 4 

- -r. from each X score in group 2, the results of the analysis would be ■ 

-. -as shown in Tabl.e 4. Note that here while the Appropriate ^COVA 
. , yields^ a---resTjrt''that is significant at £ <.0S, results of the ANORES 
: with b^- are significant at £ < . 0 2 S^^^ut^^thf\ANM 

.- . significtoti E >.J^^_l^a^-act-V-^e F for AN0RES-bj is less than : 

•' • h^lf that-^JHr^QRES^b^V'-^^^^^^^^ — . • 

In' addition, it is; possible to- duplicate, the ANCOyAr fesjalrs by 
, employing both b-p an4 b^^" to form residual scores and then applying. 
\ -Equation 19. -^The reason this^ gro.be be seen in Tables 

-3nanH 4. ANORES with ~bj^ yields the correct adjiasted SS^, and ANORES 
"with bj yields the correct adjusted SS^; Thus , ■ Equation 19 proviijes 
^ . a ratio^of the Adjusted Jtean Square Between divided by the Adjusted 
^ V Mean _Squaxe-^ desired* / ^ 

^''^ . / Several authors of experimcjital- design, texts^^^^^^ the 



■ • . • - u 

■aaiusted betWceh-grbujs sum 6£ squares in AN CO VA is-dbtkihed by sub- 
-^■tractihg. the SS^ 'for residuals using- b^^ :£rdih the SSj for resi'dukls 
■using bj, instead 0;f by direct eaieulation; Howeve.r, the actual 
. reason for the necessity of this ■ approach has not been clearly eluci- 
dated by behavioral statisticians. For example, tindquist states - 
that "A different adjusted snm ,bf squares for between -groups could 
be directly compufed as I Cfi-b.^Xi')^, using the within-groups regres- . 
sion coefficient. However, ah adjusted sum of squares fOr between- . 
groups thus computed would-be inflated by sampling error in the 
estimate (bwD of the regression coefficient empioy e d^^ah d jcoul j^ make 
the between-groups effect appear more, significant than it really is" 
(1953, p. 323J-- Although Lihdquist's conclusion is /correct ^ the 'crux'' 
oS the matter is hot simply the nature of the b;^ estimate, JBut instead 
the use of Tj^^ alone instead; of hy^ and hj together as detailed above. 

In a somewhat similar vein, it has been stated (see, for example. 
Kirk, 1968) that bj,- should not be used to adjust both the nnmprarn T— ^ 



and the- denominator of f the F ratio because this would violate' the 
independence condition necessary for ah F rat'io. While the numerator 
and. denominator must indeed be independent ^ the current argument 
shows' that this use of :b;^ to obtain ari adjusted sum of squares between 
groups also fails even to provide a numerator quantity that is dis-- 
tributed as.- a chi -square^ . ' 

In addition to explaining why the use of- bnr alone results in 

-y - - , ' ■ ■ "... 

ah inflated and inappropriately distributed betweeh-grbup sum of 
squares, the current approach also makes clear the different roles 
.of b]if and bj^. . This distinction between b^ and b^ has beeti widely 
misinterpreted (see, for example, Cohen 5 Cohen, 1975}-. In'eonTT 
tjast to Lindquist and Kirk, Cohen and Cohen err by recommending 
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use of bj^ alone, saying that to use b.. Would result in "removing 
from Y, in part^ exactly what we mean to study" G19 75 ^ p. 308). : 
Although use o£ bj does result in a lower adjusted SSg than if b^^ 
alone were used^ use of hj in the restricted model should not be 
, . viewed as removing part of "what we mean to study." Rather, it 
gives the restricted model a fair chance in that it allows the 
estimate of the regression parameter to be an optimal, least squares 
estimate, as b^ is in the full model. . 

.fh sum, although the concept of a residual score can be a' useful 
— peda^gogi-cai -tooa :f oi--explainTng the IbgTc o£""Me0VA~it has typically 
.not been utiUzed accurately. A correct SSg can be calculated by 
using- residuals, but bUly by considering both % and b^^, and hence ■ 
at least implicitly considering two sets of residuals. In terms of " 
■the residual sco-re models, ' 

Adjusted SS^ = SSECVI) - S SECII I).... • (2 0) 



However, this eliminates much. of tfte Simplicity of the j^idual " 
scores appfoath., since it requires a synthesis of ANDRES using b^j, 
and MORES using b,^. Instead of relying on ^ANQRES to' explain ANCOVA, 
an approach Utilizing ftodel cqjnparisons and least squares clarxfieT 
the ANGQVA procedure and its underlying rationale. " ■ 
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: . . table 1 ' 
Hypothetical Data to iilustrate 
AmmA - AN0RES Relationship 



Group 1 - - • Group 2 =B= 

— - :; '"^^ — . 

X ■ Y . 1 ■ ■ ■, X ^ ' ' • ' ' 

100 100 100 105 

95 98 90 ^ '109 • ^ . 

105 102 . ; .. 95 104 ' 

110 106 105 112 

105 101 „ . 95 . 100 

o ■ ' " 

90 105 rttnr 104 




'-.>■'■■•- . • . , ■ ■ • . ■ ,' ■ ■ v 
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- Table 2' =_ _ - _ 

ANGOVA and . AKGRES . Modea.s=-and--Asscrclatea' 
Error Suflis of Squares 



Model 



-Pf-. — ■- 



.ij - bwXi^' = y + 



SSE 
110.3 

170.7 

-lf&T-3- 
175.7 
114.9 

17lV7^ 



IHJC 



IB 



' " Table 3 . l 

Analysis of Variance Tables 
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ANCOVA 



Between 
Within 
foti 



60.4 

-170:7: 



1 

To"" 



MS 
.60.4 

•4 

12.3 



4.9 



ANORES . 



with b 



W 



Between^ 



Within 
Total 



-65.-3^ 



^_JL7-5r7 



10. 



-6St3- 
12.3 








w 



= 0.20 for these data 

= 9 because of the esti^iatioh of 
* 3 in f brining' the residual 



=0.09 for these data 



*£ <.0S 



20 



is 



ANCOVA 



Table 4 

Analysis bf Variance tables 
./-for Ref vise a_Ex ample- ' • ' 

SjOUTGe SS M MS 

•Betveeri 64.3 1 64.3 

__lit£t]i-in 110 ;5" 'r'~"^ ' 12.3 

. total 174.7 



F 

5.2** 



ANORES 



with b 



W 



Bet^l^e€^i— 



Within 
Tbt^l 



-8^7- 



200.0 



-1 
10 



89.7 
12.3 



7.3*** 



with T), 



BetKeen 

Within 

Total 



46;1 
174.7 



1 ; 
10 



46.1 
14.3 



3.2* 



b^ _= 0.20 for these data 

° ^^W " ^ because of the estimation of 
S in forming: the residual 

Q •_ ■ ■ ■_. . . : , ' . 

^ -6:.01 for these data 




